Large businesses, enterprises, and organizations have recently experienced extensive growth in the size and of their data and its complexity. Many large businesses rely on “legacy” data systems that may be duplicative and incompatible and that may include complex, heterogeneous mix of relational databases and files with point to point replication connections amongst them. Legacy data systems also often utilize database management systems (“DBMS”) or file management systems that cannot accommodate dynamic schema changes. These legacy data systems have become brittle because they require many successive enhancements and extensions around the data. These legacy data systems may each have different software requirements, requiring more than just one software upgrade to update the entire data system.
We have determined that the major organizing principle of data commonplace in Information Technology (“IT”) organizations needs to be fundamentally transformed to support the agility and scalability needed in today's business. One of the problems is that traditional IT topologies force businesses to choose between either having massive scale data storage or having performance and flexibility. For example, a business may have many platforms, where each platform maintains or executes a function that furthers the business's goals. Multiple of these platforms may utilize the same data, but each platform may store its own set of data, despite the fact that the data is commonly used amongst platforms. When the data is changed by one of the platforms, each legacy data system must update its data, potentially slowing down the business's computing functions. The more platforms a business has, the more time is required to update the data in each of these platforms.
However, businesses based on these legacy data systems may find it difficult and risky to continue executing their business during the transportation of data from their legacy systems to a new system. A problem faced by businesses is the major investment of funds into transporting their legacy data, only to have that data fail on the new system after major costs have been incurred. Such failures can occur because of the complexities and changes in the business priorities in the new data systems. Another reason why major transitions often fail is that there is no acceptable interim state. Processes that transport data require converting all the legacy data to the new system and then terminating the access to the legacy data system, resulting in an inability to access or operate on the legacy system. This can cause the business to be at high risk for creating data integrity issues and potentially losing the connections to the previous legacy data.